This invention relates to 3D motion adaptive decoding of video signals.
In the National TV Systems Committee (NTSC) standard for television transmission, RGB (Red Green and Blue) signals from a television camera are converted into a luma signal and a chroma signal to allow for efficient transmission. The luma signal, which is typically labeled ‘Y’, describes the brightness and maximum detail of a picture registered with the television camera. The chroma signal is a combination of two color difference signals R−Y (red minus Y) and B−Y (blue minus Y), both of which are bandwidth limited to about 1 MHz. The respective color difference signals are used to modulate the phase and amplitude of a 3.58 MHz sub-carrier signal. The phase modulation represents the hue, or specific color intended, whereas the amplitude modulation represents the intensity, or saturation, of the color difference signal. The frequency of the sub-carrier is chosen such that the chroma signal can be frequency interleaved with the luma signal to generate a single, composite signal having no energy interference between the chroma and luma signals.
When a television set receives the NTSC composite signal, the composite signal is decoded in a decoder to separate the luma and chroma signals. The chroma signal is then demodulated into R−Y and B−Y component signals that can be used to recreate RGB signals to be displayed on the television screen. Since the luma and chroma signals share the same frequency bandwidth at around 3.58 MHz, and the luma and chroma signals are not prefiltered before they are added, the luma and chroma signals cannot be fully separated in the decoding process. Thus, as a result of the crosstalk between the luma and chroma signals, various decoding artifacts are present. A number of methods have been designed to achieve a better separation of the luma and chroma signals during the decoding process in order to minimize the decoding artifacts.
Fundamentally, there are two methods for separating the luma and chroma signals. Both involve filters that are designed to discern one portion of the composite signal from the other. The first method uses a “notch/bandpass filter,” and the second method uses a “comb filter.” As will be seen below, there are many different types of comb filters that all have specific advantages and drawbacks.
The notch filter is designed to pass all frequencies of the composite signal, except the frequency of the chroma signal centered at 3.58 MHz. As a result, the chroma signal is removed, but so is also a corresponding portion of the luma signal, resulting in a loss of high frequency luma information. The notch filter is used in parallel with a bandpass filter, which passes only the frequencies in the narrow chroma band and outputs the chroma signal and the high frequency portion of the luma signal. In summary, the notch/bandpass filter method has the advantages of being a simple, low-cost approach with little or no loss of vertical resolution (that is, low frequencies). The drawbacks are that the luma resolution is lost and that heavy display artifacts result when the high frequency luma is treated as chroma (which is known as a “rainbow pattern” artifact), and when chroma is treated as luma (which is known as dot crawls).
The comb filtering methods are based on that the sub-carrier phase is designed to reverse by 180 degrees between two adjacent image lines (and thereby also between two adjacent frames, since there is an odd number of lines within a frame). There are two major types of comb filtering: line combing and frame combing. In both methods, the basic concept involves adding or subtracting two identical signals that have the chroma phase reversed by 180 degrees. When adding the signals, the chroma signal is canceled out and the luma signal is output, and when subtracting the signals, the luma information is lost and the chroma signal is output.
In line combing, one line, two lines or three lines can be used. When a single line is used, the incoming video signal is delayed by one line and the corresponding delayed and un-delayed pixels are added and subtracted, respectively, to obtain the luma and chroma signals. When two lines are used, weighted addition and subtraction of one-line delayed, two-line delayed and un-delayed pixels are used to obtain the luma and chroma signals, respectively. When three lines are used, a correlation is determined between lines 1 and 2, and lines 2 and 3, respectively, and the comb filtering is performed on the two lines that have the best correlation. The three-line adaptive comb filter is often referred to as a 2D adaptive comb filter. All line combing methods produce better results than the notch/bandpass filter method, but still show cross-color, vertical resolution loss, and dot crawls when a single line color is present.
Frame combing is similar to line combing, but instead of comparing two adjacent lines within a single frame, two lines with the same position within two adjacent frames are used for the combing, thereby extending the comb filtering into the temporal domain. However, frame combing can only be performed for still portions of an image. If motion is present, then it is necessary to revert back to 2D-combing. The method of using the motion detection to change comb filtering from line combing to frame combing is referred to as motion-adaptive decoding, or 3D adaptive decoding. The 3D adaptive decoding keeps full horizontal and vertical resolution without dot crawls and cross color for still images. However, motion artifacts, such as large area dot crawls and ghosting shadows, can be present
In the 3D adaptive decoding, the comb filtering is switched between the 3D and 2D comb filtering methods on a pixel-by-pixel basis. If no motion is present, then the 3D comb filter is applied, and if motion is present, then the 2D comb filter is applied. Hence, there is a need to minimize the motion artifacts by accurately detecting whether there is “true motion” between two frames, or whether the perceived motion is “false motion” due to, for example, jitter or noise.